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Abstract 

For many years people have speculated that electroencephalographic activity or other electrophysiological measures of brain function 
might provide a new non-muscular channel for sending messages and commands to the external world - a brain-computer interface (BCI). 
Over the past 15 years, productive BCI research programs have arisen. Encouraged by new understanding of brain function, by the advent of 
powerful low-cost computer equipment, and by growing recognition of the needs and potentials of people with disabilities, these programs 
concentrate on developing new augmentative communication and control technology for those with severe neuromuscular disorders, such as 
amyotrophic lateral sclerosis, brainstem stroke, and spinal cord injury. The immediate goal is to provide these users, who may be completely 
paralyzed, or ‘locked in’, with basic communication capabilities so that they can express their wishes to caregivers or even operate word 
processing programs or neuroprostheses. Present-day BCIs determine the intent of the user from a variety of different electrophysiological 
signals. These signals include slow cortical potentials. P300 potentials, and mu or beta rhythms recorded from the scalp, and cortical neuronal 
activity recorded by implanted electrodes. They are translated in real-time into commands that operate a computer display or other device. 
Successful operation requires that the user encode commands in these signals and that the BCI derive the commands from the signals. Thus, 
the user and the BCI system need to adapt to each other both initially and continually so as to ensure stable performance. Current BCIs have 
maximum information transfer rates up to 10-25 bits/min. This limited capacity can be valuable for people whose severe disabilities prevent 
them from using conventional augmentative communication methods. At the same time, many possible applications of BCI technology, such 
as neuroprosthesis control, may require higher information transfer rates. Future progress will depend on: recognition that BCI research and 
development is an interdisciplinary problem, involving neurobiology, psychology, engineering, mathematics, and computer science; identi¬ 
fication of those signals, whether evoked potentials, spontaneous rhythms, or neuronal firing rates, that users are best able to control 
independent of activity in conventional motor output pathways; development of training methods for helping users to gain and maintain 
that control; delineation of the best algorithms for translating these signals into device commands; attention to the identification and 
elimination of artifacts such as electromyographic and electro-oculographic activity; adoption of precise and objective procedures for 
evaluating BCI performance; recognition of the need for long-term as well as short-term assessment of BCI performance; identification 
of appropriate BCI applications and appropriate matching of applications and users; and attention to factors that affect user acceptance of 
augmentative technology, including ease of use, cosmesis, and provision of those communication and control capacities that are most 
important to the user. Development of BCI technology will also benefit from greater emphasis on peer-reviewed research publications and 
avoidance of the hyperbolic and often misleading media attention that tends to generate unrealistic expectations in the public and skepticism 
in other researchers. With adequate recognition and effective engagement of all these issues, BCI systems could eventually provide an 
important new communication and control option for those with motor disabilities and might also give those without disabilities a supple¬ 
mentary control channel or a control channel useful in special circumstances. © 2002 Elsevier Science Ireland Ltd. All rights reserved. 
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1. Introduction 

1.1. Options for restoring function to those with motor 
disabilities 

Many different disorders can disrupt the neuromuscular 
channels through which the brain communicates with and 
controls its external environment. Amyotrophic lateral 
sclerosis (ALS), brainstem stroke, brain or spinal cord 
injury, cerebral palsy, muscular dystrophies, multiple 
sclerosis, and numerous other diseases impair the neural 
pathways that control muscles or impair the muscles them¬ 
selves. They affect nearly two million people in the United 
States alone, and far more around the world (Ficke, 1991; 
NABMRR, 1992; Murray and Lopez, 1996; Carter, 1997). 
Those most severely affected may lose all voluntary muscle 
control, including eye movements and respiration, and may 
be completely locked in to their bodies, unable to commu¬ 
nicate in any way. Modern life-support technology can 
allow most individuals, even those who are locked-in, to 
live long lives, so that the personal, social, and economic 
burdens of their disabilities are prolonged and severe. 

In the absence of methods for repairing the damage done 
by these disorders, there are 3 options for restoring function. 
The first is to increase the capabilities of remaining pathways. 
Muscles that remain under voluntary control can substitute 
for paralyzed muscles. People largely paralyzed by massive 
brainstem lesions can often use eye movements to answer 
questions, give simple commands, or even operate a word 
processing program; and severely dysarthric patients can use 
hand movements to produce synthetic speech (e.g. Damper et 
al., 1987; LaCourse and Hladik, 1990; Chen et ah. 1999; 
Kubota et ah, 2000). The second option is to restore function 
by detouring around breaks in the neural pathways that 
control muscles. In patients with spinal cord injury, electro¬ 
myographic (EMG) activity from muscles above the level of 
the lesion can control direct electrical stimulation of paral¬ 
yzed muscles, and thereby restore useful movement (Hoffer 
et ah, 1996; Kilgore et ah, 1997; Ferguson et ah, 1999). 

The final option for restoring function to those with motor 
impairments is to provide the brain with a new, non-muscular 
communication and control channel, a direct brain-computer 
interface (BCI) for conveying messages and commands to 
the external world. A variety of methods for monitoring brain 
activity might serve as a BCI. These include, besides electro¬ 
encephalography (EEG) and more invasive electrophysiolo- 
gical methods, magnetoencephalography (MEG), positron 
emission tomography (PET), functional magnetic resonance 
imaging (fMRI), and optical imaging. Flowever, MEG, PET, 
fMRI, and optical imaging are still technically demanding 
and expensive. Furthermore, PET, fMRI, and optical 
imaging, which depend on blood flow, have long time 
constants and thus are less amenable to rapid communica¬ 
tion. At present, only EEG and related methods, which have 
relatively short time constants, can function in most environ¬ 
ments, and require relatively simple and inexpensive equip¬ 


ment, offer the possibility of a new non-muscular 
communication and control channel, a practical BCI. 

1.2. The fourth application of the EEG 

In the 7 decades since Hans Berger’s original paper 
(Berger, 1929), the EEG has been used mainly to evaluate 
neurological disorders in the clinic and to investigate brain 
function in the laboratory; and a few studies have explored 
its therapeutic possibilities (e.g. Travis et al., 1975; Kuhl- 
man, 1978; Elbert et al.. 1980; Rockstroh et al., 1989; Rice 
et al., 1993; Sterman, 2000). Over this time, people have 
also speculated that the EEG could have a fourth applica¬ 
tion, that it could be used to decipher thoughts, or intent, so 
that a person could communicate with others or control 
devices directly by means of brain activity, without using 
the normal channels of peripheral nerves and muscles. This 
idea has appeared often in popular fiction and fantasy (such 
as the movie ‘Firefox’ in which an airplane is controlled in 
part by the pilot’s EEG (Thomas, 1977)). However, EEG- 
based communication attracted little serious scientific atten¬ 
tion until recently, for at least 3 reasons. 

First, while the EEG reflects brain activity, so that a 
person’s intent could in theory be detected in it. the resolu¬ 
tion and reliability of the information detectable in the spon¬ 
taneous EEG is limited by the vast number of electrically 
active neuronal elements, the complex electrical and spatial 
geometry of the brain and head, and the disconcerting trial- 
to-trial variability of brain function. The possibility of 
recognizing a single message or command amidst this 
complexity, distortion, and variability appeared to be extre¬ 
mely remote. Second, EEG-based communication requires 
the capacity to analyze the EEG in real-time, and until 
recently the requisite technology either did not exist or 
was extremely expensive. Third, there was in the past little 
interest in the limited communication capacity that a first- 
generation EEG-based BCI was likely to offer. 

Recent scientific, technological, and societal events have 
changed this situation. First, basic and clinical research has 
yielded detailed knowledge of the signals that comprise the 
EEG. For the major EEG rhythms and for a variety of 
evoked potentials, their sites and mechanisms of origin 
and their relationships with specific aspects of brain func¬ 
tion, are no longer wholly obscure. Numerous studies have 
demonstrated correlations between EEG signals and actual 
or imagined movements and between EEG signals and 
mental tasks (e.g. Keirn and Aunon, 1990; Lang et al., 
1996; Pfurtscheller et al., 1997; Anderson et al., 1998; 
Altenmiiller and Gerloff, 1999; McFarland et al.. 2000a). 
Thus, researchers are in a much better position to consider 
which EEG signals might be used for communication and 
control, and how they might best be used. Second, the extre¬ 
mely rapid and continuing development of inexpensive 
computer hardware and software supports sophisticated 
online analyses of multichannel EEG. This digital revolu¬ 
tion has also led to appreciation of the fact that simple 
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communication capacities (e.g. ‘Yes’ or ‘No’, ‘On’ or ‘Off’) 
can be configured to serve compfex functions (e.g. word 
processing, prosthesis control). Third, greatly increased 
societal recognition of the needs and potential of people 
with severe neuromuscular disorders like spinal cord injury 
or cerebral palsy has generated clinical, scientific, and 
commercial interest in better augmentative communication 
and control technology. Development of such technology is 
both the impetus and the justification for current BCI 
research. BCI technology might serve people who cannot 
use conventional augmentative technologies; and these 
people could find even the limited capacities of first-genera¬ 
tion BCI systems valuable. 

In addition, advances in the development and use of elec- 
trophysiological recording methods employing epidural, 
subdural, or intracortical electrodes offer further options. 
Epidural and subdural electrodes can provide EEG with 
high topographical resolution, and intracortical electrodes 
can follow the activity of individual neurons (Schmidt, 
1980; Ikeda and Shibbasaki, 1992; Heetderks and Schmidt, 
1995; Levine et al., 1999, 2000; Wolpaw et al.. 2000a). 
Furthermore, recent studies show that the firing rates of an 
appropriate selection of cortical neurons can give a detailed 
picture of concurrent voluntary movement (e.g. Georgopou- 
los et al., 1986; Schwartz, 1993; Chapin et al., 1999; Wess- 
berg et al., 2000). Because these methods are invasive, the 
threshold for their clinical use would presumably be higher 
than for methods based on scalp-recorded EEG activity, and 
they would probably be used mainly by those with extremely 
severe disabilities. At the same time, they might support 
more rapid and precise communication and control than the 
scalp-recorded EEG. 

1.3. The present review 

This review summarizes the current state of BCI research 
with emphasis on its application to the needs of those with 
severe neuromuscular disabilities. In order to address all 
current BCI research, it includes approaches that use stan¬ 
dard scalp-recorded EEG as well as those that use epidural, 
subdural, or intracortical recording. While all these present- 
day BCIs use electrophysiological methods, the basic prin¬ 
ciples of BCI design and operation discussed here should 
apply also to BCIs that use other methods to monitor brain 
activity (e.g. MEG, fMRI). The next sections describe the 
essential elements of any BCI and the several categories of 
electrophysiological BCIs, review current research, consider 
prospects for the future, and discuss the issues most impor¬ 
tant for further BCI development and application. 

2. Definition and features of a BCI 

2.1. Dependent and independent BCIs 

A BCI is a communication system in which messages or 
commands that an individual sends to the external world do 


not pass through the brain’s normal output pathways of 
peripheral nerves and muscles. For example, in an EEG- 
based BCI the messages are encoded in EEG activity. A 
BCI provides its user with an alternative method for acting 
on the world. BCIs fall into two classes: dependent and 
independent. 

A dependent BCI does not use the brain’s normal output 
pathways to carry the message, but activity in these path¬ 
ways is needed to generate the brain activity (e.g. EEG) that 
does carry it. For example, one dependent BCI presents the 
user with a matrix of letters that flash one at a time, and the 
user selects a specific letter by looking directly at it so that 
the visual evoked potential (VEP) recorded from the scalp 
over visual cortex when that letter flashes is much larger that 
the VEPs produced when other letters flash (Sutter, 1992). 
In this case, the brain’s output channel is EEG, but the 
generation of the EEG signal depends on gaze direction, 
and therefore on extraocular muscles and the cranial nerves 
that activate them. A dependent BCI is essentially an alter¬ 
native method for detecting messages carried in the brain’s 
normal output pathways: in the present example, gaze direc¬ 
tion is detected by monitoring EEG rather than by monitor¬ 
ing eye position directly. While a dependent BCI does not 
give the brain a new communication channel that is inde¬ 
pendent of conventional channels, it can still be useful (e.g. 
Sutter and Tran, 1992). 

In contrast, an independent BCI does not depend in any 
way on the brain’s normal output pathways. The message is 
not carried by peripheral nerves and muscles, and, further¬ 
more, activity in these pathways is not needed to generate 
the brain activity (e.g. EEG) that does carry the message. 
For example, one independent BCI presents the user with a 
matrix of letters that flash one at a time, and the user selects 
a specific letter by producing a P300 evoked potential when 
that letter flashes (Farwell and Donchin, 1988; Donchin et 
al., 2000). In this case, the brain’s output channel is EEG, 
and the generation of the EEG signal depends mainly on the 
user’s intent, not on the precise orientation of the eyes 
(Sutton et al., 1965; Donchin, 1981; Fabiani et al., 1987; 
Polich, 1999). The normal output pathways of peripheral 
nerves and muscles do not have an essential role in the 
operation of an independent BCI. Because independent 
BCIs provide the brain with wholly new output pathways, 
they are of greater theoretical interest than dependent BCIs. 
Furthermore, for people with the most severe neuromuscu¬ 
lar disabilities, who may lack all normal output channels 
(including extraocular muscle control), independent BCIs 
are likely to be more useful. 

2.2. BCI use is a skill 

Most popular and many scientific speculations about 
BCIs start from the ‘mind-reading’ or ‘wire-tapping’ 
analogy, the assumption that the goal is simply to listen in 
on brain activity as reflected in electrophysiological signals 
and thereby determine a person’s wishes. This analogy 
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ignores the essential and central fact of BCI development 
and operation. A BCI changes electrophysiological signals 
from mere reflections of central nervous system (CNS) 
activity into the intended products of that activity: messages 
and commands that act on the world. It changes a signal 
such as an EEG rhythm or a neuronal firing rate from a 
reflection of brain function into the end product of that 
function: an output that, like output in conventional neuro¬ 
muscular channels, accomplishes the person’s intent. A BCI 
replaces nerves and muscles and the movements they 
produce with electrophysiological signals and the hardware 
and software that translate those signals into actions. 

The brain’s normal neuromuscular output channels 
depend for their successful operation on feedback. Both 
standard outputs such as speaking or walking and more 
specialized outputs such as singing or dancing require for 
their initial acquisition and subsequent maintenance contin¬ 
ual adjustments based on oversight of intermediate and final 
outcomes (Salmoni, 1984; Ghez and Krakauer, 2000). 
When feedback is absent from the start, motor skills do 
not develop properly; and when feedback is lost later on, 
skills deteriorate. 

As a replacement for the brain’s normal neuromuscular 
output channels, a BCI also depends on feedback and on 
adaptation of brain activity based on that feedback. Thus, a 
BCI system must provide feedback and must interact in a 
productive fashion with the adaptations the brain makes in 
response to that feedback. This means that BCI operation 
depends on the interaction of two adaptive controllers: the 
user’s brain, which produces the signals measured by the 
BCI; and the BCI itself, which translates these signals into 
specific commands. 

Successful BCI operation requires that the user develop 
and maintain a new skill, a skill that consists not of proper 
muscle control but rather of proper control of specific elec¬ 
trophysiological signals; and it also requires that the BCI 
translate that control into output that accomplishes the 
user’s intent. This requirement can be expected to remain 
even when the skill does not require initial training. In the 
independent BCI described above, the P300 generated in 
response to the desired letter occurs without training. Never¬ 
theless, once this P300 is engaged as a communication chan¬ 
nel, it is likely to undergo adaptive modification (Rosenfeld, 
1990; Coles and Rugg, 1995), and the recognition and 
productive engagement of this adaptation will be important 
for continued successful BCI operation. 

That the brain’s adaptive capacities extend to control of 
various electrophysiological signal features was initially 
suggested by studies exploring therapeutic applications of 
the EEG. They reported conditioning of the visual alpha 
rhythm, slow potentials, the mu rhythm, and other EEG 
features (Wyricka and Sterman, 1968; Dalton, 1969; 
Black et al., 1970; Nowles and Kamiya, 1970; Black, 
1971, 1973; Travis et al., 1975; Kuhlman, 1978; Rockstroh 
et al., 1989) (reviewed in Neidermeyer (1999)). These 
studies usually sought to produce an increase in the ampli¬ 


tude of a specific EEG feature. Because they had therapeutic 
goals, such as reduction in seizure frequency, they did not 
try to demonstrate rapid bidirectional control, that is, the 
ability to increase and decrease a specific feature quickly 
and accurately, which is important for communication. 
Nevertheless, they suggested that bidirectional control is 
possible, and thus justified and encouraged efforts to 
develop EEG-based communication. In addition, studies 
in monkeys showed that the firing rates of individual corti¬ 
cal neurons could be operantly conditioned, and thus 
suggested that cortical neuronal activity provides another 
option for non-muscular communication and control (Fetz 
and Finocchio, 1975; Wyler and Burchiel, 1978; Wyler et 
al., 1979; Schmidt, 1980). 

At the same time, these studies did not indicate to what 
extent the control that people or animals develop over these 
electrophysiological phenomena depends on activity in 
conventional neuromuscular output channels (e.g. Dewan, 
1967). While studies indicated that conditioning of hippo¬ 
campal activity did not require mediation by motor 
responses (Dalton, 1969; Black, 1971), the issue was not 
resolved for other EEG features or for cortical neuronal 
activity. This question of independent control of the various 
electrophysiological signal features used in current and 
contemplated BCIs is important both theoretically and prac¬ 
tically, and arises at multiple points in this review. 

2.3. The parts of a BCI 

Like any communication or control system, a BCI has 
input (e.g. electrophysiological activity from the user), 
output (i.e. device commands), components that translate 
input into output, and a protocol that determines the onset, 
offset, and timing of operation. Fig. 1 shows these elements 
and their principal interactions. 

2.3.1. Signal acquisition 

In the BCIs discussed here, the input is EEG recorded 
from the scalp or the surface of the brain or neuronal activity 
recorded within the brain. Thus, in addition to the funda¬ 
mental distinction between dependent and independent 
BCIs (Section 2.1 above), electrophysiological BCIs can 
be categorized by whether they use non-invasive (e.g. 
EEG) or invasive (e.g. intracortical) methodology. They 
can also be categorized by whether they use evoked or 
spontaneous inputs. Evoked inputs (e.g. EEG produced by 
flashing letters) result from stereotyped sensory stimulation 
provided by the BCI. Spontaneous inputs (e.g. EEG rhythms 
over sensorimotor cortex) do not depend for their generation 
on such stimulation. There is, presumably, no reason why a 
BCI could not combine non-invasive and invasive methods 
or evoked and spontaneous inputs. In the signal-acquisition 
part of BCI operation, the chosen input is acquired by the 
recording electrodes, amplified, and digitized. 
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Fig. 1. Basic design and operation of any BCI system. Signals from the brain are acquired by electrodes on the scalp or in the head and processed to extract 
specific signal features (e.g. amplitudes of evoked potentials or sensorimotor cortex rhythms, firing rates of cortical neurons) that reflect the user's intent. These 
features are translated into commands that operate a device (e.g. a simple word processing program, a wheelchair, or a neuroprosthesis). Success depends on 
the interaction of two adaptive controllers, user and system. The user must develop and maintain good correlation between his or her intent and the signal 
features employed by the BCI; and the BCI must select and extract features that the user can control and must translate those features into device commands 
correctly and efficiently. 


2.3.2. Signal processing: feature extraction 

The digitized signals are then subjected to one or more of a 
variety of feature extraction procedures, such as spatial filter¬ 
ing, voltage amplitude measurements, spectral analyses, or 
single-neuron separation. This analysis extracts the signal 
features that (hopefully) encode the user’s messages or 
commands. BCIs can use signal features that are in the 
time domain (e.g. evoked potential amplitudes or neuronal 
firing rates) or the frequency domain (e.g. mu or beta-rhythm 
amplitudes) (Farwell and Donchin, 1988; Lopes da Silva and 
Mars, 1987; Parday et al., 1996; Lopes da Silva, 1999; 
Donchin et al., 2000; Kennedy et al., 2000; Wolpaw et al., 
2000b; Pfurtscheller et al., 2000a; Penny et al., 2000; Kostov 
and Polak, 2000). A BCI could conceivably use both time- 
domain and frequency-domain signal features, and might 
thereby improve performance (e.g. Schalk et al., 2000). 


In general, the signal features used in present-day BCIs 
reflect identifiable brain events like the firing of a specific 
cortical neuron or the synchronized and rhythmic synaptic 
activation in sensorimotor cortex that produces a mu 
rhythm. Knowledge of these events can help guide BCI 
development. The location, size, and function of the cortical 
area generating a rhythm or an evoked potential can indicate 
how it should be recorded, how users might best learn to 
control its amplitude, and how to recognize and eliminate 
the effects of non-CNS artifacts. It is also possible for a BCI 
to use signal features, like sets of autoregressive parameters, 
that correlate with the user’s intent but do not necessarily 
reflect specific brain events. In such cases, it is particularly 
important (and may be more difficult) to ensure that the 
chosen features are not contaminated by EMG, electro¬ 
oculography (EOG), or other non-CNS artifacts. 
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2.3.3. Signal processing: the translation algorithm 

The first part of signal processing simply extracts specific 
signal features. The next stage, the translation algorithm, 
translates these signal features into device commands- 
orders that carry out the user’s intent. This algorithm 
might use linear methods (e.g. classical statistical analyses 
(Jain et al., 2000) or nonlinear methods (e.g. neural 
networks). Whatever its nature, each algorithm changes 
independent variables (i.e. signal features) into dependent 
variables (i.e. device control commands). 

Effective algorithms adapt to each user on 3 levels. First, 
when a new user first accesses the BCI the algorithm adapts 
to that user’s signal features. If the signal feature is mu- 
rhythm amplitude, the algorithm adjusts to the user’s 
range of mu-rhythm amplitudes; if the feature is P300 
amplitude, it adjusts to the user’s characteristic P300 ampli¬ 
tude; and if the feature is the firing rate of a single cortical 
neuron, it adjusts to the neuron’s characteristic range of 
firing rates. A BCI that possesses only this first level of 
adaptation, i.e. that adjusts to the user initially and never 
again, will continue to be effective only if the user’s perfor¬ 
mance is very stable. However, EEG and other electrophy- 
siological signals typically display short- and long-term 
variations linked to time of day, hormonal levels, immediate 
environment, recent events, fatigue, illness, and other 
factors. Thus, effective BCIs need a second level of adapta¬ 
tion: periodic online adjustments to reduce the impact of 
such spontaneous variations. A good translation algorithm 
will adjust to these variations so as to match as closely as 
possible the user’s current range of signal feature values to 
the available range of device command values. 

While they are clearly important, neither of these first two 
levels of adaptation addresses the central fact of effective 
BCI operation: its dependence on the effective interaction of 
two adaptive controllers, the BCI and the user’s brain. The 
third level of adaptation accommodates and engages the 
adaptive capacities of the brain. As discussed in Section 
2.2, when an electrophysiological signal feature that is 
normally merely a reflection of brain function becomes 
the end product of that function, that is, when it becomes 
an output that carries the user’s intent to the outside world, it 
engages the adaptive capacities of the brain. Like activity in 
the brain’s conventional neuromuscular communication and 
control channels, BCI signal features will be affected by the 
device commands they are translated into: the results of BCI 
operation will affect future BCI input. In the most desirable 
(and hopefully typical) case, the brain will modify signal 
features so as to improve BCI operation. If, for example, the 
feature is mu-rhythm amplitude, the correlation between 
that amplitude and the user’s intent will hopefully increase 
over time. An algorithm that incorporates the third level of 
adaptation could respond to this increase by rewarding the 
user with faster communication. It would thereby recognize 
and encourage the user’s development of greater skill in this 
new form of communication. On the other hand, excessive 
or inappropriate adaptation could impair performance or 


discourage further skill development. Proper design of this 
third level of adaptation is likely to prove crucial for BCI 
development. Because this level involves the interaction of 
two adaptive controllers, the user’s brain and the BCI 
system, its design is among the most difficult problems 
confronting BCI research. 

2.3.4. The output device 

For most current BCIs, the output device is a computer 
screen and the output is the selection of targets, letters, or 
icons presented on it (e.g. Farwell and Donchin, 1988; 
Wolpaw et al., 1991; Perelmouter et al., 1999; Pfurtscheller 
et al., 2000a). Selection is indicated in various ways (e.g. the 
letter flashes). Some BCIs also provide additional, interim 
output, such as cursor movement toward the item prior to its 
selection (e.g. Wolpaw et al., 1991; Pfurtscheller et al., 
2000a). In addition to being the intended product of BCI 
operation, this output is the feedback that the brain uses to 
maintain and improve the accuracy and speed of commu¬ 
nication. Initial studies are also exploring BCI control of a 
neuroprosthesis or orthesis that provides hand closure to 
people with cervical spinal cord injuries (Lauer et al., 
2000; Pfurtscheller et al., 2000b). In this prospective BCI 
application, the output device is the user’s own hand. 

2.3.5. The operating protocol 

Each BCI has a protocol that guides its operation. This 
protocol defines how the system is turned on and off, 
whether communication is continuous or discontinuous, 
whether message transmission is triggered by the system 
(e.g. by the stimulus that evokes a P300) or by the user, 
the sequence and speed of interactions between user and 
system, and what feedback is provided to the user. 

Most protocols used in BCI research are not completely 
suitable for BCI applications that serve the needs of people 
with disabilities. Most laboratory BCIs do not give the user 
on/off control; the investigator turns the system on and off. 
Because they need to measure communication speed and 
accuracy, laboratory BCIs usually tell their users what 
messages or commands to send. In real life the user picks 
the message. Such differences in protocol can complicate 
the transition from research to application. 

3. Present-day BCIs 

While many studies have described electrophysiological 
or other measures of brain function that correlate with 
concurrent neuromuscular outputs or with intent and 
might therefore function in a BCI system, relatively few 
peer-reviewed articles have described human use of systems 
that satisfy the BCI definition given in Section 2.1 and illu¬ 
strated in Fig. 1, systems that give the user control over a 
device and concurrent feedback from the device. These 
studies are reviewed here. Studies from the vast group 
describing phenomena that might serve as the basis for a 
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BCI are mentioned only when they relate directly to actual 
BCI systems. 

Present-day BCIs fall into 5 groups based on the electro- 
physiological signals they use. The first group, those using 
VEPs, are dependent BCIs, i.e. they depend on muscular 
control of gaze direction. The other 4 groups, those using 
slow cortical potentials, P300 evoked potentials, mu and 
beta rhythms, and cortical neuronal action potentials, are 
believed to be independent BCIs (Section 2.1), though this 
belief remains to some extent an assumption still in need of 
complete confirmation. 

3.1. Visual evoked potentials 

In the 1970s, Jacques Vidal used the term ‘brain-compu¬ 
ter interface’ to describe any computer-based system that 
produced detailed information on brain function. This early 
usage was broader than current usage, which applies the 
term BCI only to those systems that support communication 
and control by the user. Nevertheless, in the course of his 
work, Vidal developed a system that satisfied the current 
definition of a dependent BCI (Vidal, 1973, 1977). This 
system used the VEP recorded from the scalp over visual 
cortex to determine the direction of eye gaze (i.e. the visual 
fixation point), and thus to determine the direction in which 
the user wished to move a cursor. 

Sutter (1992) described a similar BCI system called the 
brain response interface (BRI). It uses the VEPs produced 
by brief visual stimuli and recorded from the scalp over 
visual cortex. The user faces a video screen displaying 64 
symbols (e.g. letters) in an 8X8 grid and looks at the 
symbol he or she wants to select. Subgroups of these 64 
symbols undergo an equiluminant red/green alternation or 
a fine red/green check pattern alternation 40-70 times/s. 
Each symbol is included in several subgroups, and the entire 
set of subgroups is presented several times. Each subgroup’s 
VEP amplitude about 100 ms after the stimulus is computed 
and compared to a VEP template already established for the 
user. From these comparisons, the system determines with 
high accuracy the symbol that the user is looking at. A 
keyboard interface gives access to output devices. Normal 
volunteers can use it to operate a word processing program 
at 10-12 words/min. In users whose disabilities cause 
uncontrollable head and neck muscle activity, scalp EMG 
can impede reliable VEP measurement and reduce perfor¬ 
mance. For one such user, a man with ALS, this problem 
was solved by placing a strip of 4 epidural electrodes over 
visual cortex. With this implant, he could communicate 10- 
12 words/min (Sutter, 1984, 1992). 

Middendorf et al. (2000) reported another method for 
using VEPs to determine gaze direction. Several virtual 
buttons appear on a screen and flash at different rates. The 
user looks at a button and the system determines the 
frequency of the photic driving response over visual cortex. 
When this frequency matches that of a button, the system 
concludes that the user wants to select it. 


These VEP-based communication systems depend on the 
user’s ability to control gaze direction. Thus, they perform 
the same function as systems that determine gaze direction 
from the eyes themselves, and can be categorized as depen¬ 
dent BCI systems. They show that the EEG can yield precise 
information about concurrent motor output, and might prove 
superior to other methods for assessing gaze direction. It is 
possible that VEP amplitude in these systems reflects atten¬ 
tion as well as gaze direction (e.g. Teder-Salejarvi et al., 
1999), and thus that they may be to some extent independent 
of neuromuscular function. 

3.2. Slow cortical potentials 

Among the lowest frequency features of the scalp- 
recorded EEG are slow voltage changes generated in cortex. 
These potential shifts occur over 0.5-10.0 s and are called 
slow cortical potentials (SCPs). Negative SCPs are typically 
associated with movement and other functions involving 
cortical activation, while positive SCPs are usually asso¬ 
ciated with reduced cortical activation (Rockstroh et al., 
1989; Birbaumer, 1997). In studies over more than 30 
years, Birbaumer and his colleagues have shown that people 
can learn to control SCPs and thereby control movement of 
an object on a computer screen (Elbert et al., 1980, Birbau¬ 
mer et al., 1999, 2000). This demonstration is the basis for a 
BCI referred to as a ‘thought translation device’ (TTD). The 
principal emphasis has been on developing clinical applica¬ 
tion of this BCI system. It has been tested extensively in 
people with late-stage ALS and has proved able to supply 
basic communication capability (Kubler, 2000). 

In the standard format (Fig. 2A), EEG is recorded from 
electrodes at the vertex referred to linked mastoids. SCPs 
are extracted by appropriate filtering, corrected for EOG 
activity, and fed back to the user via visual feedback from 
a computer screen that shows one choice at the top and one 
at the bottom. Selection takes 4 s. During a 2 s baseline 
period, the system measures the user’s initial voltage 
level. In the next 2 s, the user selects the top or bottom 
choice by decreasing or increasing the voltage level by a 
criterion amount. The voltage is displayed as vertical move¬ 
ment of a cursor and final selection is indicated in a variety 
of ways. The BCI can also operate in a mode that gives 
auditory or tactile feedback (Birbaumer et al., 2000). 
Users train in several 1-2 h sessions/week over weeks or 
months. When they consistently achieve accuracies >75%, 
they are switched to a language support program (LSP). 

The LSP (Perelmouter et al., 1999; Perelmouter and 
Birbaumer, 2000) enables the user to choose a letter or letter 
combination by a series of two-choice selections. In each 
selection, the choice is between selecting or not selecting a 
set of one or more letters. The first two selections choose 
between the two halves of the alphabet, the next two 
between the two quarters of the selected half, and so on 
until a single letter is chosen. A backup or erase option is 
provided. With this program, users who have two-choice 
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Fig. 2. Present-day human BCI system types. (Modified from Kiibler et al. (2001).) A-C are non-invasive methods, D is invasive. (A) SCP BCI. Scalp EEG is 
recorded from the vertex. Users learn to control SCPs to move a cursor toward a target (e.g. a desired letter or icon) at the bottom (more positive SCP) or top 
(more negative SCP) of a computer screen (Kiibler et al., 2001;Birbaumer et al., 1999, 2000). (B) P300 BCI. A matrix of possible choices is presented on a 
screen and scalp EEG is recorded over the centroparietal area while these choices flash in succession. Only the choice desired by the user evokes a large P300 
potential (i.e. a positive potential about 300 ms after the flash) (Farwell and Donchin, 1988; Donchin et al., 2000). (C) Sensorimotor rhythm BCI. Scalp EEG is 
recorded over sensorimotor cortex. Users control the amplitude of a 8-12 Hz mu rhythm (or a 18-26 Hz beta rhythm) to move a cursor to a target at the top of 
the screen or to a target at the bottom (or to additional targets at intermediate locations). Frequency spectra (top) for top and bottom targets show that control is 
clearly focused in the mu-rhythm frequency band. Sample EEG traces (bottom) also indicate that the mu rhythm is prominent when the target is at the top and 
minimal when it is at the bottom (Wolpaw et al., 1991, 2000b; McFarland et al., 1997a). (D) Cortical neuronal BCI. Cone electrodes implanted in motor cortex 
detect action potentials of single cortical neurons (traces). Users learn to control neuronal firing rate(s) to move a cursor to select letters or icons on a screen 
(Kennedy and Bakay, 1998; Kennedy et al., 2000). 


accuracies of 65-90% can write 0.15-3.0 letters/min, or 2- 
36 words/h. While these rates are low, the LSP has proved 
useful to and highly valued by people who cannot use 
conventional augmentative communication technologies. 
Furthermore, a predictive algorithm that uses the first two 
letters of a word to select the word from a lexicon that 
encompasses the user’s vocabulary can markedly increase 
the communication rate. A new protocol provides Internet 
access to one disabled user (Birbaumer et al., 2000). A 
stand-by mode allows users wearing collodium-fixed elec¬ 
trodes to access the system 24 h/day by producing a specific 
sequence of positive and negative SCPs (Kaiser et al., 


2001). This sequence is essentially a key for turning the 
BCI on and off. 


3.3. P300 evoked potentials 

Infrequent or particularly significant auditory, visual, or 
somatosensory stimuli, when interspersed with frequent or 
routine stimuli, typically evoke in the EEG over parietal 
cortex a positive peak at about 300 ms (Walter et al., 
1964; Sutton et al., 1965; Donchin and Smith, 1970). 
Donchin and his colleagues have used this ‘P300’, or 
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‘oddball’ response in a BCI (Farwell and Donchin, 1988; 
Donchin et al.. 2000). 

The user faces a 6 X 6 matrix of letters, numbers, and/or 
other symbols or commands. Every 125 ms, a single row or 
column flashes; and, in a complete trial of 12 flashes, each 
row or column flashes twice. The user makes a selection by 
counting how many times the row or column containing the 
desired choice flashes. EEG over parietal cortex is digitized, 
the average response to each row and column is computed, 
and P300 amplitude for each possible choice is computed. 
As Fig. 2B shows, P300 is prominent only in the responses 
elicited by the desired choice, and the BCI uses this effect to 
determine the user’s intent. In online experiments and 
offline simulations, a variety of different algorithms (e.g. 
stepwise discriminant analysis, discrete wavelet transform) 
for recognizing the desired choice have been evaluated, and 
the relationship between the number of trials per selection 
and BCI accuracy has been described. These analyses 
suggest that the current P300-based BCI could yield a 
communication rate of one word (i.e. 5 letters) per minute 
and also suggest that considerable further improvement in 
speed should be possible. In people with visual impair¬ 
ments, auditory or tactile stimuli might be used (Glover et 
al., 1986; Roder et al., 1996). In related work, Bayliss and 
Ballard (2000) recorded P300s in a virtual environment. 
Offline analyses suggested that single-trial P300 amplitudes 
might be used for environmental control. 

A P300-based BCI has an apparent advantage in that it 
requires no initial user training: P300 is a typical, or naive, 
response to a desired choice. At the same time, P300 and 
related potentials change in response to conditioning proto¬ 
cols (Glover et al., 1986; Miltner et al., 1988; Sommer and 
Schweinberger, 1992; Roder et al., 1996). A P300 used in a 
BCI is also likely to change over time. Studies up to the 
present have been short-term. In the long term, P300 might 
habituate (Ravden and Polich, 1999) so that BCI perfor¬ 
mance deteriorates, or it might get larger so that perfor¬ 
mance improves. Thus, appropriate adaptation by the 
translation algorithm is likely to be important for this 
BCI, as it is for others. 

3.4. Mu and beta rhythms and other activity from 
sensorimotor cortex 

In awake people, primary sensory or motor cortical areas 
often display 8-12 Hz EEG activity when they are not 
engaged in processing sensory input or producing motor 
output (Gastaut, 1952; Kozelka and Pedley, 1990; Fisch, 
1999) (reviewed in Neidermeyer, 1999). This idling activ¬ 
ity, called mu rhythm when focused over somatosensory or 
motor cortex and visual alpha rhythm when focused over 
visual cortex, is thought to be produced by thalamocortical 
circuits (Lopes da Silva, 1991; Neidermeyer, 1999). Unlike 
the visual alpha rhythm, which is obvious in most normal 
people, the mu rhythm was until quite recently found only in 
a minority (Chatrian, 1976). However, computer-based 


analyses reveal the mu rhythm in most adults (Pfurtscheller, 
1989). Such analyses also show that mu-rhythm activity 
comprises a variety of different 8-12 Hz rhythms, distin¬ 
guished from each other by location, frequency, and/or rela¬ 
tionship to concurrent sensory input or motor output. These 
mu rhythms are usually associated with 18-26 Hz beta 
rhythms. While some beta rhythms are harmonics of mu 
rhythms, some are separable from them by topography 
and/or timing, and thus are independent EEG features 
(Pfurtscheller and Berghold, 1989; Pfurtscheller, 1999; 
McFarland et al., 2000a). 

Several factors suggest that mu and/or beta rhythms could 
be good signal features for EEG-based communication. 
They are associated with those cortical areas most directly 
connected to the brain’s normal motor output channels. 
Movement or preparation for movement is typically accom¬ 
panied by a decrease in mu and beta rhythms, particularly 
contralateral to the movement. This decrease has been 
labeled ‘event-related desynchronization’ or ERD 
(Pfurtscheller and Lopes da Silva, 1999b; Pfurtscheller, 
1999). Its opposite, rhythm increase, or ‘event-related 
synchronization’ (ERS) occurs after movement and with 
relaxation (Pfurtscheller, 1999). Furthermore, and most 
relevant for BCI use, ERD and ERS do not require actual 
movement, they occur also with motor imagery (i.e. 
imagined movement) (Pfurtscheller and Neuper, 1997; 
McFarland et al., 2000a). Thus, they might support an inde¬ 
pendent BCI. Since the mid-1980s, several mu/beta rhythm- 
based BCIs have been developed. 

3.4.1. The Wadsworth BCI 

With the BCI system of Wolpaw, McFarland, and their 
colleagues (Wolpaw et al., 1991, 2000b; McFarland et al., 
1997a), people with or without motor disabilities learn to 
control mu- or beta-rhythm amplitude and use that control to 
move a cursor in one or two dimensions to targets on a 
computer screen. Fig. 2C shows the basic phenomenon. In 
this example, the user increases the amplitude of a 8-12 Hz 
mu rhythm to move a cursor to a target at the top of the 
screen or decreases it to move to a target at the bottom. 
Frequency spectra (top) for top and bottom targets show 
that control is clearly focused in the mu-rhythm frequency 
band. Sample EEG traces (bottom) also show that the mu 
rhythm is prominent with the top target and minimal with 
the bottom target. 

For each dimension of cursor control, a linear equation 
translates mu- or beta-rhythm amplitude from one or several 
scalp locations into cursor 10 times/s. Users learn over a 
series of 40 min sessions to control cursor movement. 
They participate in 2-3 sessions per week, and most (i.e. 
about 80%) acquire significant control within 2-3 weeks. In 
the initial sessions, most employ motor imagery (e.g. imagi¬ 
nation of hand movements, whole body activities, relaxa¬ 
tion, etc.) to control the cursor. As training proceeds, 
imagery usually becomes less important, and users move 
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the cursor like they perform conventional motor acts, that is, 
without thinking about the details of performance. 

While EEG from only one or two scalp locations controls 
cursor movement online, data from 64 locations covering 
the entire scalp are gathered for later offline analysis that 
defines the full topography of EEG changes associated with 
target position and helps guide improvements in online 
operation. This analysis relies largely on the measure r 2 , 
which is the proportion of the total variance in mu- or 
beta-rhythm amplitude that is accounted for by target posi¬ 
tion and thus reflects the user’s level of EEG control. The r 2 
topographical analyses (e.g. Fig. 3C) show that control is 
sharply focused over sensorimotor cortex and in the mu- 
and/or beta-rhythm frequency bands. With this control, 
users can move the cursor to answer spoken yes/no ques¬ 
tions with accuracies >95% (Miner et al., 1998; Wolpaw et 
al., 1998). They can also achieve independent control of two 
different mu- or beta-rhythm channels and use that control 
to move a cursor in two dimensions (Wolpaw and McFar¬ 
land, 1994). Recent work has concentrated on developing 
precise one-dimensional control, and on applying it to 
choosing among up to 8 different targets. Users have 
achieved information transfer rates up to 20-25 bits/min 
(McFarland et al., 2000b). 

Research with this BCI has focused on defining the topo¬ 
graphical, spectral, and temporal features of mu- and beta- 
rhythm control and on optimizing the mutually adaptive 
interactions between the user and the BCI system. Improve¬ 
ments include: spatial Alters that match the spatial frequen¬ 
cies of the user’s mu or beta rhythms (Fig. 3), autoregressive 
frequency analysis which gives higher resolution for short 
time segments and thus permits more rapid device control, 
and better selection of the constants in the equations that 
translate EEG control into device control (e.g. McFarland et 
al., 1997a,b; Ramoser et al., 1997). Recent studies have also 
explored incorporation of other EEG features into this BCI. 
In well-trained users, errors in target selection are associated 
with a positive potential centered at the vertex (Schalk et al., 

2000) . This potential might be used to recognize and cancel 
mistakes. While work to date has used cursor control as a 
prototype BCI application and has concentrated on improv¬ 
ing it, effort is also being devoted to applications like 
answering simple questions or basic word processing 
(Miner et al.. 1998; Wolpaw et al., 1998; Vaughan et al., 

2001 ) . 

3.4.2. The Graz BCI 

This BCI system is also based on ERD and ERS of mu 
and beta rhythms. Research up to the present has focused on 
distinguishing between the EEG associated with imagina¬ 
tion of different simple motor actions, such as right or left 
hand or foot movement, and thereby enabling the user to 
control a cursor or an orthotic device that opens and closes a 
paralyzed hand (Pfurtscheller et al., 1993, 2000a,b; Neuper 
et al., 1999). In the standard protocol, the user first partici¬ 
pates in an initial session to select a motor imagery para¬ 


digm. In each of a series (e.g. 160) of 5.25 s trials, the user 
imagines one of several actions (e.g. right or left hand or 
foot movement, tongue movement) while EEG from elec¬ 
trodes over sensorimotor cortex is submitted to frequency 
analysis to derive signal features (e.g. the powers in the 
frequency bands from 5 to 30 Hz). For each imagined 
action, an n-dimensional feature vector is deflned. These 
vectors establish a user-specific linear or non-linear classi- 
fler (e.g. linear discriminant analysis, distinction sensitive 
learning vector quantization (DSLVQ), or a neural network) 
that determines from the EEG which action the user is 
imagining (Pregenzer et al., 1996; Pfurtscheller et al., 
1996; Pregenzer and Pfurtscheller, 1999; Miiller-Gerking 
et al., 1999). In subsequent sessions, the system uses the 
classifier to translate the user’s motor imagery into a contin¬ 
uous output (e.g. extension of a lighted bar or cursor move¬ 
ment) or a discrete output (e.g. selection of a letter or other 
symbol), which is presented to the user as online feedback 
on a computer screen. Normally, the classiflcation algorithm 
is adjusted between daily sessions. Over 6-7 sessions with 
two-choice trials (i.e, left hand vs. right hand imagery) users 
can reach accuracies over 90%. About 90% of people can 
use this system successfully. The signal features that reflect 
motor imagery and are used by the classifier are concen¬ 
trated in the mu- and beta-rhythm bands in EEG over 
sensorimotor cortex (Pfurtscheller and Neuper, 1997). 

Current studies seek modiflcations that improve classifl¬ 
cation. These include use of parameters derived by autore¬ 
gressive frequency analysis (instead of the values for power 
in specific frequency bands) and use of alternative spatial 
filters. Additional effort has been devoted to developing 
remote control capabilities that allow the BCI to function 
in users’ homes while the classification algorithm is updated 
in the laboratory. With this remote control system, a user 
paralyzed by a mid-cervical spinal cord injury uses hand and 
foot motor imagery to control an orthosis that provides hand 
grasp. EEG over sensorimotor cortex is translated into hand 
opening and closing by autoregressive parameter estimation 
and linear discriminative classification (Obermaier et al., 
2001; Guger et al., 1999; Pfurtscheller et al., 2000b; 
Pfurtscheller and Neuper, 2001). 

3.4.3. Other systems 

Rostov and Polak (2000) report BCI control of one- and 
two-dimensional cursor movement. EEG is recorded with a 
28-electrode array and a linked-ear reference, digitized at 
200 Hz, and analyzed. Autoregressive parameters from 2 to 
4 locations are translated into cursor movements by an adap¬ 
tive logic network (Armstrong and Thomas, 1996). User 
training is important (Polak, 2000). 

Penny et al. (2000) describe a BCI that also uses EEG 
over sensorimotor cortex to control cursor movement. They 
concentrate on detecting the EEG associated with imagery 
of actions like right or left hand movements, and/or tasks 
like simple calculations. Their translation algorithm uses 
autoregressive parameters and a logistic regression model 
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Fig. 3. (A) Electrode locations used by 4 different spatial filters for EEG recorded from C3 (red). During data acquisition, all 64 electrodes are referred to the ear 
reference. For the common average reference (CAR) and Laplacian methods, EEG at the green electrodes is averaged and subtracted from EEG at C3. (B) 
Spatial band-pass. For each method, the trace shows the square root of the root-mean-square values (amplitude in |xV) of a signal that varies sinusoidally in 
amplitude across the scalp as its spatial frequency varies from 6 cm, twice the inter-electrode distance (i.e. the highest spatial frequency that would not cause 
spatial aliasing), to 60 cm (i.e. approximate head circumference). (C) Average r 2 topography, and amplitude and r 2 spectra for each spatial filter method for 
trained BCI users at the frequency and electrode location, respectively, used online. Each method is applied to the same body of data. With each method, EEG 
control (measured as r 2 , the proportion of the variance of the signal feature that is accounted for by the user’s intent) is focused over sensorimotor cortices and 
in the mu- and beta-rhythm frequency bands. The value of r 2 is highest for the CAR and large Laplacian spatial filters and lowest for the ear reference. 
(Modified from McFarland et al. 1997b.) 
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trained with a Bayesian evidence framework. They report 
user success in controlling one-dimensional cursor move¬ 
ment (Roberts and Penny, 2000). 

Other groups have explored offline a variety of 
approaches not yet tested online. Birch and Mason (Birch 
et al., 1993; Birch and Mason, 2000; Mason and Birch, 
2000) describe methods for recognizing potentials related 
to voluntary movement (VMRPs) in EEG over sensorimotor 
and supplementary motor cortices, and for using that recog¬ 
nition to control cursor movement. Their translation algo¬ 
rithm uses features extracted from the 1-4 Hz band in 
bipolar EEG channels. They have focused on recognizing 
VMRPs in ongoing EEG rather than in the EEG associated 
with externally paced trials. Thus, they are addressing a 
problem important for practical applications: detection of 
user commands without the timing cues provided by struc¬ 
tured trials. Levine et al. (2000) recorded electrocortico- 
gram (EcoG) activity from 17 patients temporarily 
implanted with 16-126 subdural electrodes prior to epilepsy 
surgery. They found topographically focused potentials 
associated with specific movements and vocalizations. 
These potentials might provide the basis for a BCI with 
multiple control channels. Pineda and Allison (Pineda et 
al., 2000, Allison et al., 2000) explored the relationship 
between single and combined movements as seen in the 
mu rhythm and the readiness potential. Babiloni et al. 
(2000) are developing a Laplacian EEG analysis and a 
signal-space projection algorithm to detect imagined move¬ 
ments in EEG over sensorimotor cortex. 

3.5. Cortical neurons 

Since the 1960s, metal microelectrodes have been used to 
record action potentials of single neurons in the cerebral 
cortices of awake animals during movements (e.g. Evarts, 
1966; Humphrey, 1986). While most studies focused on the 
relationships between this neuronal activity and simple or 
complex sensorimotor performances, a few have explored 
the capacity of animals to learn to control neuronal firing 
rates. With operant conditioning methods, several studies 
showed that monkeys could learn to control the discharge 
of single neurons in motor cortex (Fetz and Finocchio, 1975; 
Wyler and Burchiel, 1978; Wyler et al., 1979; Schmidt, 
1980). From such work came the expectation that humans, 
including many with motor disabilities, could develop simi¬ 
lar control and use it to communicate or to operate neuro- 
prostheses. 

Evaluation of this possibility was delayed by lack of 
intracortical electrodes suitable for human use and capable 
of stable long-term recording from single neurons. Conven¬ 
tional implanted electrodes induce scar tissue and/or move 
in relation to individual neurons, so that over time recording 
deteriorates or neurons come and go. In 1989, Kennedy 
described an intracortical electrode consisting of a hollow 
glass cone containing recording wires (Kennedy, 1989). 
Neural tissue or neurotrophic factors placed inside the 


cone induced cortical neurons to send processes into the 
cone so that their action potentials could be recorded (Fig. 
2D). These electrodes, implanted in motor cortices of 
monkeys and several humans nearly locked-in by ALS or 
brainstem stroke, have provided stable neuronal recordings 
for more than a year (Kennedy and Bakay, 1998; Kennedy 
et al., 2000). 

Up to now, one user has learned to control neuronal firing 
rates and uses this control to move a cursor to select icons or 
letters on a computer screen. By using neuronal activity to 
control one dimension of cursor movement and residual 
EMG control to control the other dimension and final selec¬ 
tion, communication rates up to about 3 letters/min (i.e, 
about 15 bits/min) have been achieved. While training has 
been limited by recurring illness and medication effects, the 
results have been encouraging and suggest that more rapid 
and accurate control should be possible in the future. 
Furthermore, by demonstrating this control in people who 
are almost totally paralyzed, these initial data suggest that 
cortical neurons can support an independent BCI system. 

Several laboratories have used multielectrode arrays to 
record from single neurons in motor cortex of monkeys or 
rats during learned movements (Georgopoulos et al., 1986; 
Schmidt et al., 1988; Schwartz, 1993; Donoghue and Sanes, 
1994; Heetderks and Schmidt, 1995; Nicolelis et al., 1998; 
Liu et al., 1999; Williams et al., 1999; Chapin et al., 1999; 
Isaacs et al., 2000; Wessberg et al., 2000). The results show 
that the firing rates of a set of cortical neurons can reveal the 
direction and nature of movement. At the same time, almost 
all of this work has studied neuronal activity associated with 
actual movement. It is not clear whether the same patterns of 
neuronal activity, or other stable patterns, will be present 
when the movements are not made, and, most important, 
when the animal is no longer capable of making the move¬ 
ments (due, for example, to a spinal cord injury). Limited 
data suggest that the patterns persist for at least a time in the 
absence of movement (Craggs, 1975; Chapin et al., 1999; 
Taylor and Schwartz, 2001). 


4. The future of BCI-based communication and control: 

key issues 

Non-muscular communication and control is no longer 
merely speculation. The studies reviewed in the previous 
section show that direct communication from the brain to 
the external world is possible and can serve useful purposes. 
At the same time, the reality does not yet match the fantasy 
(e.g. Thomas, 1977): BCIs are not yet able to fly airplanes 
and are not likely to be doing so anytime soon. Present 
independent BCIs in their best moments reach 25 bits/min. 
For those who have no voluntary muscle control whatsoever 
or in whom remaining control (e.g. eye movement) is weak, 
easily fatigued, or unreliable, this modest capacity may be 
valuable. For people who are totally paralyzed (e.g. by ALS, 
brainstem stroke, or severe polyneuropathy) or lack any 
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useful muscle control (e.g. due to severe cerebral palsy), a 
BCI might give the ability to answer simple questions 
quickly (i.e. 20 bits/min is 20 ‘yes/no’ questions/min, or 
one/3 s), control the environment (e.g. lights, temperature, 
television, etc.), perform slow word processing (i.e. with a 
predictive program, 25 bits/min could produce 2 words/ 
min), or even operate a neuroprosthesis (reviewed in 
Wolpaw et al., 2000a; Kiibler et al., 2001). Nevertheless, 
the future value of BCI technology will depend substantially 
on how much information transfer rate can be increased. 

BCI development is still in its earliest stages. It is not yet 
clear how far the field can or will go. What is clear is that 
how far it does go will depend on a number of crucial issues. 
These include: BCI independence from normal neuromus¬ 
cular communication channels and dependence on internal 
aspects of normal brain function; selection of signal acqui¬ 
sition methods, signal features, feature extraction methods, 
translation algorithms, output devices, and operational 
protocols; development of user training strategies; attention 
to psychological and behavioral factors that affect user 
motivation and success; adoption of standard research meth¬ 
ods and evaluation criteria; choice of applications and user 
groups; and the largely unknown capacities and limitations 
of non-muscular communication channels. 

4.1. Independence from neuromuscular output channels 

While a dependent BCI, which simply reflects activity in 
conventional neuromuscular output channels, can be useful 
(e.g. Sutter, 1984, 1992), the future importance of BCI tech¬ 
nology will hinge on the extent to which its function can be 
independent of conventional neuromuscular output chan¬ 
nels. The BCIs in Fig. 2 are thought to be independent, 
but this issue is yet to be completely settled. 

The generally successful application of an SCP-based 
BCI to people with late-stage ALS who lack almost all 
voluntary movement is persuasive evidence that SCPs can 
support independent BCIs (Birbaumer et al., 1999, 2000). 
The P300-based BCI is also likely to be independent 
(Donchin et al., 2000). P300s are believed to reflect the 
significance of the stimulus, that is, in the case of the 
P300- based BCI, whether it is the choice that the user 
wants to select. At the same time, however, the visual 
stimuli needed to elicit P300 may depend to some degree 
on control of eye gaze (Michalski, 1999; Teder-Salejarvi et 
al., 1999; Nobre et al., 2000). Available data suggest that mu 
and beta rhythms from sensorimotor cortex can support 
independent BCIs. These rhythms are affected by motor 
imagery in the absence of movement (e.g. McFarland et 
al., 2000a). Furthermore, mu- or beta-rhythm based cursor 
control does not depend on activity in cranial or limb 
muscles (Vaughan et al., 1998). Finally, mu rhythm-based 
BCI operation was achieved by a user almost totally locked 
in by ALS (Miner et al.. 1996). 

The cortical and subcortical neuronal activity that accom¬ 
panies voluntary movement is in part a function of the 


proprioceptive and other sensory feedback that occurs 
during that movement (e.g. Houk and Rymer, 1981). It is 
not yet clear to what extent users can produce this activity or 
comparably controlled activity without actual movement, 
nor to what extent other sensory modalities, e.g. vision or 
audition, can substitute effectively for the somatosensory 
feedback associated with normal voluntary motor function. 
Initial studies (Craggs, 1975; Kennedy and Bakay, 1998; 
Chapin et al., 1999; Kennedy et al., 2000; Taylor and 
Schwartz, 2001; Serruya et al., 2002) suggest that neuronal 
activity can function without movement, but the long-term 
stability of this function is not yet established. 

4.2. Degree of dependence on normal brain function 

While BCIs based on SCPs, P300s, mu and beta rhythms, 
or cortical neuronal activity may not require voluntary 
muscle control, they certainly depend to some degree on 
normal brain function. Each of these electrophysiological 
phenomena reflects the combined function of cortical and 
subcortical areas. Impairments of cortex (e.g. with ALS or 
stroke), basal ganglia or other subcortical areas that interact 
with cortex (e.g. with cerebral palsy) or loss of ascending 
sensory input (e.g. with brainstem stroke or spinal cord 
injury) could affect the user’s ability to achieve control of 
cortical potentials, mu or beta rhythms, or cortical neurons. 
Thus, the ability to use BCIs and the best choice among the 
different BCIs, are likely to differ among users. Studies that 
evaluate specific BCIs in specific user groups are needed, 
and should include long-term assessments of performance, 

4.3. Non-CNS artifacts 

Muscle activation and eye movement can contribute to 
the electrical activity recorded from the scalp (Anderer et 
al., 1999; Croft and Barry, 2000). At frontal, temporal, and 
occipital locations particularly, EMG and/or EOG can 
exceed EEG, even in the characteristic EEG frequency 
bands (McFarland et al., 1997a; Goncharova et al., 2000). 
While EMG and EOG may serve in their own rights in 
augmentative communication systems (ten Kate and Hepp, 
1989; Tecce et al., 1998; Barreto et al., 2000), in the context 
of BCI research they are simply artifacts that must be recog¬ 
nized and addressed. They can mislead investigators by 
mimicking actual EEG-based control and/or can impede 
measurement of the EEG features used for control. For 
example, frontalis muscle EMG can dominate the beta- or 
mu-rhythm frequency range at frontal locations, and eye- 
blinks can affect the theta- or even mu-rhythm range at 
frontal or central locations (e.g. McFarland et al., 1997a; 
Goncharova et al., 2000). Thus, a user might control BCI 
output by raising his eyebrows or blinking her eyes; or such 
activity might obscure the user’s actual EEG control. 

Spectral and topographic analyses can usually detect non- 
CNS artifacts. However, studies that look only at one 
frequency band or scalp location, or rely completely on 
signal features, such as autoregressive parameters, that 
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may be complex functions of EEG and non-CNS activity, 
risk contamination by non-CNS artifacts. These artifacts can 
produce misleading results that lead to erroneous conclu¬ 
sions about the characteristics, capacities, and limitations of 
EEG-based BCIs, and can thereby impede research and 
development. A recent preliminary study (Lauer et al., 
1999) that purported to show control of a neuroprosthesis 
by EEG over frontal cortex illustrates this danger. Subse¬ 
quent work showed that frontalis EMG was largely or 
wholly responsible for the control (Lauer et al., 2000). 
The spectral and topographical analyses that can detect 
such artifacts and the procedures that can prevent them 
from affecting BCI operation or misleading investigators 
are extremely important in BCI research (Wolpaw et al., 
2000a). 

4.4. Signal features 

Most current BCIs use electrophysiological signal 
features that represent brain events that are reasonably 
well-defined anatomically and physiologically. These 
include rhythms reflecting oscillations in particular neuronal 
circuits (e.g. mu or beta rhythms from sensorimotor cortex), 
potentials evoked from particular brain regions by particular 
stimuli (e.g. VEPs or P300s), or action potentials produced 
by particular cortical neurons (Kennedy et al., 2000). A few 
are exploring signal features, such as autoregressive para¬ 
meters, that bear complex and uncertain relationships to 
underlying brain events (Lopes da Silva and Mars, 1987; 
Parday et al., 1996; Lopes da Silva, 1999). 

The special characteristics and capacities of each signal 
feature will largely determine the extent and nature of its 
usefulness. SCPs are, as their name suggests, slow. They 
develop over 300 ms to several seconds. Thus, if an SCP- 
based BCI is to exceed a bit rate of one every 1-2 s, users 
will need to produce more than two SCP levels at one loca¬ 
tion, and/or control SCPs at several locations independently. 
Initial studies suggest that such control may be possible 
(Kotchoubey et al., 1996, 1997; Hardman et al., 1997). 
While mu and beta rhythms have characteristic frequencies 
of 8-12 and 18-26 Hz, respectively, change in mu- or beta- 
rhythm amplitude appears to have a latency of about 0.5 s 
(Wolpaw et al., 1997; Pfurtscheller, 1999; Pfurtscheller and 
Lopes da Silva, 1999a). On the other hand, users are 
certainly able to provide more than two amplitude levels, 
and can achieve independent control of different rhythms 
(Wolpaw and McFarland, 1994; Vaughan et al., 1999). 
Projecting from results to date, a mu/beta rhythm BCI 
might select among 4 or more choices every 2-3 s (McFar¬ 
land et al., 2000b). While the possibility for distinguishing 
more than two amplitude ranges from VEPs or P300 poten¬ 
tials has not been explored, these potentials can be evoked in 
partially overlapping series of trials, so that selection rate 
can be increased (Donchin et al., 2000). Alternatively or in 
addition, selection rate might be increased if users could 
learn to control shorter-latency evoked potentials (e.g. 


Finley, 1984). The firing rates of individual cortical 
neurons, if they prove to be independently controllable in 
the absence of the concurrent motor outputs and sensory 
inputs that normally accompany and reflect their activity, 
might support quite high information transfer rates. 

The key determinant of a signal feature’s value is its 
correlation with the user’s intent, that is, the level of volun¬ 
tary control the user achieves over it. Users are likely to 
differ in the signal features they can best control. In 3 
users nearly locked in by ALS, Kiibler (2000) found that 
one used a positive SCP, another a relatively fast negative¬ 
positive SCP shift, and a third a P300. Once developed, 
these strategies were extremely resistant to change. Particu¬ 
larly early in training, BCI systems should be able to iden¬ 
tify, accommodate, and encourage the signal features best 
suited to each user. 

User training may be the most important and least under¬ 
stood factor affecting the BCI capabilities of different signal 
features. Up to now, researchers have usually assumed that 
basic learning principles apply. However, BCI signal 
features are not normal or natural brain output channels. 
They are artificial output channels created by BCI systems. 
It is not yet clear to what extent these new artificial outputs 
will observe known conditioning principles. For example, 
mu rhythms and other features generated in sensorimotor 
cortex, which is directly involved in motor output, may 
prove more useful than alpha rhythms generated in visual 
or auditory cortex, which is strongly influenced by sensory 
input. The success of neuronally based BCI methods will 
presumably also vary from area to area. Initial efforts have 
focused on neurons in motor cortex (Kennedy, 1989; 
Kennedy and Bakay, 1998). While this focus is logical, 
other cortical areas and even subcortical areas warrant 
exploration (e.g. Chapin et al., 1999). For example, in a 
user paralyzed by a peripheral nerve or muscle disorder, 
the activity of spinal cord motoneurons controlling specific 
muscles, detected by implanted electrodes (e.g. Nuwer, 
1999; Mushahwar et al., 2000), might prove most useful 
for communication and control. 

4.5. Signal processing: feature extraction 

The performance of a BCI, like that of other communica¬ 
tion systems, depends on its signal-to-noise ratio. The goal 
is to recognize and execute the user’s intent, and the signals 
are those aspects of the recorded electrophysiological activ¬ 
ity that correlate with and thereby reveal that intent. The 
user’s task is to maximize this correlation; and the system’s 
first task is to measure the signal features accurately, i.e, to 
maximize the signal-to-noise ratio. When the features are 
mu rhythms from sensorimotor cortex, noise includes visual 
alpha rhythms, and when the features are the firing rates of 
specific neurons, noise includes activity of other neurons. Of 
particular importance for EEG-based BCIs is the detection 
and/or elimination of non-CNS activity, such as EMG from 
cranial or facial muscles and EOG (Section 4.3). 
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Feature extraction methods can greatly affect signal-to- 
noise ratio. Good methods enhance the signal and reduce 
CNS and non-CNS noise. This is most important and diffi¬ 
cult when the noise is similar to the signal. For example, 
EOG is of more concern than EMG for a BCI that uses SCPs 
as signal features (Birbaumer et al., 1990), because EOG 
and SCPs have overlapping frequency ranges; and for the 
same reason EMG is of more concern than EOG for BCIs 
that use beta rhythms (Goncharova et al., 2000). 

A variety of options for improving BCI signal-to-noise 
ratios are under study. These include spatial and temporal 
filtering techniques, signal averaging, and single-trial recog¬ 
nition methods. Much work up to now has focused on show¬ 
ing by offline data analyses that a given method will work. 
Careful comparisons of alternative methods are also essen¬ 
tial. A statistical measure useful in such comparisons is r 2 , 
the proportion of the total variance in the signal feature that 
is accounted for by the user’s intent. Alternative feature 
extraction methods can be compared in terms of r 2 . (At 
the same time, of course, it is essential to insure that a 
high r 2 is not being achieved by non-CNS activity such as 
EMG.) Finally, any method must ultimately be shown to be 
useful for actual online operation. 

Spatial filters derive signal features by combining data 
from two or more locations so as to focus on activity with 
a particular spatial distribution. The simplest spatial filter is 
the bipolar derivation, which derives the first spatial deriva¬ 
tive and thereby enhances differences in the voltage gradient 
in one direction. The Laplacian derivation is the second 
derivative of the instantaneous spatial voltage distribution, 
and thereby emphasizes activity in radial sources immedi¬ 
ately below the recording location (Zhou, 1993; Nunez et 
al., 1997). It can be computed by combining the voltage at 
the location with the voltages of surrounding electrodes (e.g. 
Hjorth, 1991; Nunez et al., 1994). As the distance to the 
surrounding electrodes decreases, the Laplacian becomes 
more sensitive to voltage sources with higher spatial 
frequencies (i.e. more localized sources) and less sensitive 
to those with lower spatial frequencies (i.e. more broadly 
distributed sources). 

The choice of a spatial filter can markedly affect the 
signal-to-noise ratio of a BCI that uses mu and beta rhythms 
(McFarland et al.. 1997b). Fig. 3 displays the results for 4 
different spatial filters applied to the same data from trained 
users. It shows that a Laplacian with an inter-electrode 
distance of 6 cm (or a common average reference method) 
provides a much higher signal-to-noise ratio (measured as 
r 2 ) than does either a Laplacian with a distance of 3 cm or a 
monopolar derivation. On the other hand, a spatial filter best 
suited for mu and beta rhythms, which are relatively loca¬ 
lized, would probably not be the best choice for measure¬ 
ment of SCPs or P300s, which are more broadly distributed 
over the scalp. 

Laplacian and common average reference spatial biters 
apply a bxed set of weights to a linear combination of 
channels (i.e. electrode locations). Both use weights that 


sum to zero so that the result is a difference and the spatial 
biter has high-pass characteristics. Other spatial biters are 
available. Principal components, independent components, 
and common spatial patterns analyses are alternative meth¬ 
ods for deriving weights for a linear combination of chan¬ 
nels (e.g. Miiller-Gerking et al., 1999; Jung et al., 2000). In 
these methods, the weights are determined by the data. Prin¬ 
cipal components analysis, which produces orthogonal 
components, may not be appropriate for separation of signal 
features from overlapping sources. Independent compo¬ 
nents analysis can, in principle, distinguish between mu 
rhythms from such sources (Makeig et al., 2000). These 
methods have yet to be compared to simpler spatial biters 
like the Laplacian, in which the channel weights are data- 
independent. 

Appropriate temporal bitering can also enhance signal-to- 
noise ratios (e.g. McFarland et al., 1997a). Oscillatory 
signals like the mu rhythm can be measured by the inte¬ 
grated output of a band-pass biter (Pfurtscheller and Arani- 
bar, 1979) or by the amplitude in specibc spectral bands of 
Fourier or autoregressive analysis (Marple, 1987). Because 
BCIs must provide relatively rapid user feedback and 
because signals may change rapidly, frequency analysis 
methods (e.g. band-pass biters or autoregressive methods) 
that need only relatively short time segments may be super¬ 
ior to methods like Fourier analysis that need longer 
segments. The choice of temporal bitering method, particu¬ 
larly for research studies, should also consider the need to 
detect non-CNS artifacts. A single band-pass biter cannot 
identify a broad-band artifact like EMG; a representative set 
of such biters is needed. Similarly, when autoregressive 
parameters are used as signal features (e.g. Pfurtscheller et 
al., 1998), additional spectral-band analyses are needed to 
detect artifacts like EMG. For SCP recording, the focus on 
extremely-low frequency activity (e.g. high-pass biter cutoff 
<0.1 Hz) requires attention to eye-movement (Gratton et 
al., 1983) and other low-frequency artifacts like those due 
to ampliber drift or changes in skin resistance (e.g. with 
sweating). 

The signal-to-noise ratios of evoked time-domain signals 
like P300 can be enhanced by averaging. The accompanying 
loss in communication rate may be minimized by overlap¬ 
ping the trials (e.g. Donchin et al.. 2000). A variety of 
methods have been proposed for detecting signals in single 
trials (Arpaia et al., 1989; Hufschmidt et al., 1990; Birch et 
al., 1993; Lange et al., 1995; Mineva and Popivanov, 1996; 
Schlogl et al., 1997; Samar et al., 1999; Birch and Mason, 
2000). These methods have yet to be extensively applied in 
BCI research. Thus, their potential usefulness is unclear. 

Invasive methods using epidural, subdural, or intracorti- 
cal electrodes might give better signal-to-noise ratios than 
noninvasive methods using scalp electrodes. At the same 
time, the threshold for their use will presumably be higher. 
They will be used only when they can provide communica¬ 
tion clearly superior to that provided by noninvasive meth¬ 
ods, or when they are needed to avoid artifacts or other 
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problems that can impede noninvasive methods (e.g. 
uncontrollable head and neck EMG in a user with cerebral 
palsy). 

4.6. Signal processing: translation algorithms 

BCI translation algorithms convert independent vari¬ 
ables, that is, signal features such as rhythm amplitudes or 
neuronal firing rates, into dependent variables (i.e. device 
control commands). Commands may be continuous (e.g. 
vertical cursor movements) or discrete (e.g. letter selection). 
They should be as independent of each other (i.e. orthogo¬ 
nal) as possible, so that, for example, vertical cursor move¬ 
ment and horizontal cursor movement do not depend on 
each other. The success of a translation algorithm is deter¬ 
mined by the appropriateness of its selection of signal 
features, by how well it encourages and facilitates the user’s 
control of these features, and by how effectively it translates 
this control into device commands. If the user has no control 
(i.e. if the user’s intent is not correlated with the signal 
features), the algorithm can do nothing, and the BCI will 
not work. If the user has some control, the algorithm can do 
a good or bad job of translating that control into device 
control. 

Initial selection of signal features for the translation algo¬ 
rithm can be based on standard guidelines (e.g. the known 
locations and temporal and spatial frequencies of mu and 
beta rhythms) supplemented by operator inspection of initial 
topographical and spectral data from each user (Rockstroh 
et al., 1984; McFarland et al., 1997a). These methods may 
be supplemented or even wholly replaced by automated 
procedures. For example, Pregenzer et al. (1996), used the 
learning vector quantizer (LVQ) to select optimal electrode 
positions and frequency bands for each user. 

Extant BCIs use a variety of translation algorithms, 
ranging from linear equations, to discriminant analysis, to 
neural networks (e.g. Wolpaw et al., 2000b; Pfurtscheller et 
al., 2000a; Kostov and Polak, 2000). In the simplest case, in 
which only a single signal feature is used, the output of the 
translation algorithm can be a simple linear function of the 
feature value (e.g. a linear function of mu-rhythm ampli¬ 
tude). The algorithm needs to use appropriate values for the 
intercept and the slope of this function (McFarland et al., 
1997a). If the command is vertical cursor movement, the 
intercept should ensure that upward and downward move¬ 
ment are equally possible for the user. Ramoser et al. (1997) 
found that the mean value of the signal feature over some 
interval of immediately preceding performance provides a 
good estimate of the proper intercept. The slope determines 
the scale of the command (e.g. the speed of cursor move¬ 
ment). When a single signal feature is used to select among 
more than two choices, the slope also affects the relative 
accessibility of the choices (e.g. McFarland et al., 1999, 
2000b). A wide variety of more complex translation algo¬ 
rithms are possible. These include supervised learning 
approaches such as linear discriminate analysis (e.g. Jain 


et al., 2000) and non-linear discriminate analysis (e.g. an 
adaptive logic network (Kostov and Polak, 2000). 

The evaluation of a translation algorithm reduces to 
determining how well it accomplishes the 3 levels of adap¬ 
tation described in Section 2.3.3: initial adaptation to the 
individual user; continuing adaptation to spontaneous 
changes in the user’s performance (e.g. fatigue, level of 
attention); and continuing adaptation that encourages and 
guides the user’s adaptation to the BCI (i.e. user training). 
Up to the present, most evaluations have concentrated on the 
first and simplest level of adaptation. In these evaluations, 
alternative algorithms are applied offline to a body of data 
gathered from one or more users. Typically, portions of the 
data are used to set the parameters of the algorithm (e.g. to 
train the neural network), which is then applied to the rest of 
the data (i.e. the test data) (Pregenzer et al., 1996; Miiller- 
Gerking et al., 1999). The algorithm is rated according to the 
accuracy with which it derives the user’s intent from the test 
data. While such evaluations are convenient and certainly 
valuable in making gross distinctions between algorithms, 
they do not take into account spontaneous changes in the 
signal features, nor can they assess user adaptation to the 
algorithm. 

The second level of adaptation - continual adjustments 
for spontaneous changes in signal features - can be 
addressed by offline analysis that mimics the online situa¬ 
tion. that is, if adaptation is based on earlier data and applied 
to later data (e.g. Ramoser et al., 1997). This analysis needs 
substantial bodies of data gathered over substantial periods 
of time, so that all major kinds of spontaneous variation can 
be assessed. The need for this second level of adaptation 
tends to favor simpler algorithms. Parameter adaptation is 
likely to be more difficult and more vulnerable to instabil¬ 
ities for complex algorithms like those using neural 
networks or non-linear equations, than it is for simple algo¬ 
rithms like those using linear equations with relatively few 
variables. 

The third level of adaptation - adaptation to the user’s 
adaptation to the BCI system - is not accessible to offline 
evaluation. Because this level responds to and affects the 
continual interactions between the user and the BCI, it can 
only be assessed online. The goal of this adaptation is to 
induce the user to develop and maintain the highest possible 
level of correlation between his or her intent and the signal 
features that the BCI employs to decipher that intent. The 
algorithm can presumably accomplish these aims by 
rewarding better performance - by moving the cursor or 
selecting the letter more quickly when the signal feature 
has a stronger correlation with intent. At the same time, 
such efforts at shaping user performance risk making the 
task too difficult. As with acquisition of conventional skills, 
anxiety, frustration, or fatigue can degrade performance 
(e.g. Dibartolo et al., 1997). Particularly in the first stages 
of training, the user is easily overwhelmed by the difficulty 
of the task. User success may correlate with self-perception 
of brain states, and may be promoted by procedures that 
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increase this perception (Lang and Kotchoubey, 2000). 
Because the translation algorithm’s adaptations are likely 
to shape the user’s adaptations, and because users are likely 
to differ from one another, the selection of methods for this 
third level of adaptation inevitably requires prolonged 
online studies in large numbers of representative users. 

This level of adaptation might also help address the 
problem of artifacts, such as EMG or EOG for scalp EEG 
or extraneous neuronal activity for neuronal recording. It 
may be possible to induce the user to reduce or eliminate 
such artifacts by making them impediments to performance. 
Thus, a specific measure of EMG activity, like amplitude in 
a high frequency band at a suitable location, could be moni¬ 
tored, and, by exceeding a criterion value, could halt BCI 
operation. 

The mutual adaptation of user and BCI is likely to be 
important even for BCIs that use signal features (e.g. 
P300 evoked potentials, or mu- or beta-rhythm amplitude 
changes accompanying specific motor imagery) that are 
already present in users at the very beginning of training. 
Once these features are used for communication and 
control, they can be expected to change. Like the activity 
responsible for the brain’s neuromuscular outputs, these 
electrophysiological phenomena are likely to be continu¬ 
ously adjusted on the basis of feedback. The process of 
mutual adaptation of the user to the system and the system 
to the user is likely to be a fundamental feature of the opera¬ 
tion of any BCI system (McFarland et al., 1998; Siniatchkin 
et al., 2000). Thus, the value of starting from signal features 
that are already correlated with specific intents in the naive 
user (e.g. P300) is an empirical issue. That is, does BCI 
training that begins with such features ultimately lead to 
faster and more accurate communication and control than 
does training that begins with other features? 

These adaptations by the translation algorithm may be 
more difficult in actual BCI applications than in the labora¬ 
tory. In the usual laboratory situation, user intent is defined by 
the research protocol (i.e. the user is told what to select), so 
that the translation algorithm knows whether the user’s selec¬ 
tions are correct and can use this knowledge in its adapta¬ 
tions. In real life, the user decides what to select, so that the 
translation algorithm does not have this knowledge and adap¬ 
tation is therefore more difficult. Possible solutions are to 
configure applications so as to insure fairly predictable sets 
of past intents, to incorporate calibration routines that consist 
of series of trials with defined intents, and/or to include meth¬ 
ods for error correction (e.g. a backspace key) that permit the 
translation algorithm to assume that all or most final selec¬ 
tions are correct. Unsupervised learning approaches, like 
cluster or principal components analysis, which can be 
trained without knowledge of correct results, might also be 
effective (Miiller-Gerking et al.. 1999; Jung et al., 2000). 

4.7. Operating protocols 

Ideally, a BCI would be available at all times. Such avail¬ 


ability might be provided either by always translating input 
into output, or by a BCI-based on/off switch. Continuous 
translation could produce much unintended, meaningless, or 
random output. One research group is now evaluating this 
option (Birch and Mason, 2000; Mason and Birch, 2000). 
Another possibility is a BCI-based on/off key, such as a 
distinctive pattern of signal features that is extremely unli¬ 
kely to occur spontaneously (e.g. a specific pattern of SCP 
shifts (Birbaumer et al., 2000; Kaiser et al., 2001). 

In voice communication, the speaker controls when 
words are said and the rate at which they are said. In 
contrast, present-day BCIs usually control timing and rate 
and indicate them to the user by visual or auditory means. 
To some extent, the system control of timing and rate results 
from the requirements of research: to assess BCI perfor¬ 
mance it is necessary to know user intent, and the simplest 
way to do this is to have the system tell the user what to 
communicate and when. In actual BCI applications, control 
of timing and rate might be vested wholly or partially in the 
user (e.g. McFarland et al., 1999). 

4.8. Applications and users 

Practical applications depend first on speed and accuracy. 
Current BCIs are suitable for basic environmental control 
(e.g. temperature, lights, television), for answering yes/no 
questions, and for word processing at slow rates (e.g. 1-2 
words/min). They might also operate devices like a wheel¬ 
chair. In this application, in which errors are dangerous, the 
system could be configured to ensure very high accuracy at 
the expense of speed. These BCIs might also operate simple 
neuroprostheses or orthoses, like those providing hand grasp 
to people with cervical spinal cord injuries (Lauer et al., 
2000; Pfurtscheller et al., 2000b). 

At the same time, while current BCIs might provide such 
functions, most potential users have better conventional 
options. Those who retain control of only a single muscle 
(e.g. eyebrow, finger flexor, diaphragm) can often use it for 
communication and control that is faster and more accurate 
than that provided by current BCIs. Thus, immediate users 
will be mainly those who lack all muscle control or whose 
remaining control is easily fatigued or otherwise unreliable. 
They include those who are totally paralyzed (e.g. by ALS 
or brainstem stroke) or have movement disorders (e.g. 
severe cerebral palsy) that abolish muscle control. Conven¬ 
tional augmentative communication methods may have 
little to offer them, so that even the simplest BCI-based 
communication, like the ability to say ‘yes’ or ‘no’, could 
be valuable. Recent data indicate that their incidence of 
depression is not necessarily higher than that of the general 
population (e.g. Robbins et al., 2002). This suggests that, if 
their communication and control needs can be satisfied, they 
can lead enjoyable and productive lives. 

Efforts to provide BCI-based communication to users 
who are locked-in may encounter several difficult issues. 
First, a user’s lack of conventional communication ability 
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can make it hard to assess his or her cognition or even 
consciousness, and may impede the operator/user interac¬ 
tions important in initial BCI training. Kotchoubey et al. 
(2002) describe a set of event-related potential-based tests 
designed to assess cortical sensory and cognitive processing 
in such users and evaluate their capacity for mastering BCI 
use. Second, the deficits that abolish all voluntary muscle 
control may also impair user control of the signal features 
used by a BCI. The loss of cortical neurons that can occur 
with ALS or the extensive cortical and/or subcortical 
damage typically associated with severe cerebral palsy 
may affect generation or control of the rhythms, evoked 
potentials, or neuronal activity used for BCI-based commu¬ 
nication. Damage to prefrontal cortex (e.g. with multiple 
sclerosis, Parkinson’s disease, or ALS) can impair attention 
and thereby adversely affect BCI use (Rockstroh et al., 
1989; Muller et al., 1997). Third, motivational factors can 
be critically important (Birbaumer et al., 2000). Changes in 
an individual’s physical environment or social interactions 
can greatly affect the extent of BCI use. Effective BCI appli¬ 
cation in clinical situations requires careful and continual 
assessment of quality of life. Standard quality of life 
measures may not be appropriate for people who are 
severely paralyzed. Their emotional and psychological 
well being does not necessarily worsen as motor function 
declines (e.g. Robbins et al., 2001). In sum, BCI applica¬ 
tions require expertise in and attention to a complex set of 
human factors. 

People who retain minimal voluntary movement might 
use hybrid systems that combine BCI-based control with 
conventional control (e.g. Kennedy et al., 2000). BCIs 
might also serve those whose communication and control 
capacities are impaired by aphasias, apraxias, or autism 
(Birbaumer, 1999). 

If BCI speed and accuracy can be substantially improved, 
the range of applications and the number of potential users 
would greatly increase. At the same time, speed and accu¬ 
racy are not the only important factors. The extent to which 
BCI use can be integrated with other activities is crucial. Up 
to now, most BCIs have been tested in the laboratory with 
the users totally involved in their operation. A few studies 
have explored BCI integration into normal life. The Wads¬ 
worth BCI can be used to answer spoken questions, and the 
Tuebingen TTD to communicate user-chosen messages 
(Miner et al., 1998; Birbaumer et al., 1999). Much more 
information concerning such integration with other brain 
functions is needed. 

BCI success will hinge also on the extent to which opera¬ 
tion is standardized. Most present BCIs operate in the 
laboratory with expert oversight. Even the Tuebingen 
TTD, which has been taken out of the laboratory, still 
requires frequent adjustment by skilled personnel (Kiibler 
et al., 2001; Kaiser et al., 2001). If BCIs are to function in 
homes or long-term care facilities, this dependence must be 
greatly reduced. 

Especially important in determining the practical value of 


BCI systems will be their success in satisfying the user (e.g. 
De Foa and Loeb, 1991; Scherer and Lane, 1997; Stroh 
Wuolle et al., 1999). Satisfaction will not depend only, or 
perhaps even mainly, on speed and accuracy. The level of 
user acceptance of a new technology depends on more than 
the theoretical value of the technology, and is often unpre¬ 
dictable or even inexplicable. Kiibler et al. (1999) found that 
several users preferred slow selection of single letters to 
faster computer-aided word completion because letter-by- 
letter selection, which was completely under their control, 
made them feel more independent. Ease of use is also 
important. Factors such as the need for constantly wearing 
an electrode cap or constantly confronting a particular 
visual display can have tremendous influence. Cosmesis is 
often crucial; that is, how the system looks and how the user 
looks while employing it. Users are likely to differ consid¬ 
erably in their needs and desires, and BCI success will 
depend in large part on recognition and accommodation of 
this reality. The primary emphasis should be on identifying 
and providing those BCI applications most desired by the 
user. 

4.9. Research methods and standards 

BCI research is an interdisciplinary endeavor. The 
phenomena that comprise BCI signal features arise in the 
brain and reflect its anatomy, chemistry, and physiology. 
BCIs perform signal processing, and depend on computer 
hardware and software. They incorporate adaptation 
routines that depend on learning principles and on other 
human factors like attention, motivation, and fatigue. BCI 
outputs control devices that have specific electronic and/or 
mechanical characteristics and provide feedback that 
engages the user’s perceptive and reactive capacities. 
Finally, BCI operating protocols orchestrate operation in 
accord with the characteristics of the inputs, feature extrac¬ 
tion methods, translation algorithms, and outputs. Thus, BCI 
research involves neurobiology, psychology, engineering, 
applied mathematics, and computer science. Success 
depends on expertise in all these disciplines and on effective 
interactions between them. 

While all BCI research programs share the same goal - 
rapid and accurate communication and control - they differ 
widely not only in their inputs, feature extraction methods, 
translation algorithms, outputs, and operating protocols, but 
also in their immediate objectives. Some focus on specific 
applications like word processing or neuroprosthesis 
control, while others concentrate on establishing general 
features of BCI design and operation and use a prototype 
application like cursor control to do so. Whatever their 
objectives, all need hardware and software that can acquire 
electrophysiological input with sufficient rate and precision, 
translate it into output quickly enough to control devices 
with acceptable delays, and manage adaptive interactions 
between user and system. Because their purpose is research, 
they also need to store complete data for later evaluation. 
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Furthermore, if their efforts are to be productive and their 
results credible, BCI research programs must adhere to 
certain principles in experimental design, data evaluation, 
and documentation and dissemination of results. 

4.9.1. Assessment of inter- and intra-user variations 

Users are likely to differ greatly in the prominence and 

stability of specific signal features, and in their capacities 
for initially demonstrating or acquiring and subsequently 
maintaining control over these features. Users with disabil¬ 
ities are likely to display even more variation. Thus, BCI 
methods should be tested in more than one or a few users, 
and the test populations should include users with relevant 
disabilities. Intra-individual variation is an equally impor¬ 
tant issue. Those few research programs that have acquired 
long-term data have found that marked variations in perfor¬ 
mance typically occur over minutes, hours, days, weeks, and 
months. Thus, data should be gathered from each user many 
times over substantial periods. Furthermore, appropriate and 
comprehensive statistical tests should be applied. Simply 
describing the single best result or the performance for a 
few sessions is not enough. 

4.9.2. Online validation of offline analyses 

Offline analyses of data stored during BCI operation are 
not by themselves sufficient for assessing and comparing 
alternative signal features, feature extraction routines, trans¬ 
lation algorithms, etc. While they can suggest which meth¬ 
ods are likely to work best online, they cannot predict the 
short- or long-term effects of differences among methods in 
the user feedback. Methods that appear promising in offline 
analyses must ultimately be validated by extensive online 
testing over prolonged periods in adequate numbers of 
users, and this testing should incorporate to the greatest 
extent possible appropriate internal (i.e. intra-individual) 
and/or external (i.e. inter-individual) controls. 

4.9.3. Assessment of both user performance and system 
performance 

Effective assessment of BCI performance requires two 
levels of evaluation: the user and the system. The user must 
control the signal features, and the system must recognize 
that control and translate it into device control effectively and 
consistently. User performance can be defined as the level of 
correlation between user intent and the signal feature(s) the 
BCI employs to recognize that intent. One useful measure of 
this correlation is r 2 (Section 4.5). Perfect correlation 
produces an r 2 value of 1.00. As illustrated in Fig. 3, this 
measure proved very useful in choosing the best spatial filter 
method for extracting mu- or beta-rhythm signal features 
(McFarland et al., 1997b). It is also valuable for selecting 
the electrode location and frequency band used to determine 
mu- or beta-rhythm amplitude (e.g. Wolpaw et al., 2000b). 

Evaluation of system performance has two parts: perfor¬ 
mance in a specific application, assessed as speed and/or 
accuracy, and theoretical performance, measured as infor¬ 


mation transfer rate. Up to now, most studies have simply 
reported the accuracy and/or speed for specific applications. 
These data are certainly important. At the same time, they 
are affected by the characteristics of the application and the 
success with which the system interfaces the user’s control 
of the signal features with that application. Thus, they make 
comparisons between different studies difficult, and they do 
not reveal what might theoretically be done with the degree 
of control that the user has. 

The standard method for measuring communication and 
control systems is information transfer rate, or bit rate. It is 
the amount of information communicated per unit time. 
Derived from Shannon and Weaver (1964) (summarized 
in Pierce, 1980), this measure incorporates both speed and 
accuracy in a single value. Fig. 4 shows the relationship 
between accuracy and bit rate for different numbers of 
choices. Bit rate is shown both as bits/trial (i.e. bits/selec¬ 
tion), and as bits/min when 12 selections are made per min 
(a rate comparable to that of several current BCIs (e.g. 
Birbaumer et al., 2000; Donchin et al.. 2000; Pfurtscheller 
et al., 2000a; Wolpaw et al., 2000b)). For example, the bit 
rate of a BCI that selects between two choices with 90% 
accuracy is equal to that of a BCI that selects among 4 
choices with 65% accuracy. The great importance of accu¬ 
racy, shown in Fig. 4, has often not received proper recogni¬ 
tion in BCI research. With two choices, 90% accuracy is 



Fig. 4. Information transfer rate in bits/trial (i.e. bits/selection) and in bits/ 
min (for 12 trials/min) when the number of possible choices (i.e. N) is 2, 4, 
8, 16, or 32. As derived from Pierce (1980) (and originally from Shannon 
and Weaver, 1964), if a trial has N possible choices, if each choice has the 
same probability of being the one that the user desires, if the probability ( P ) 
that the desired choice will actually be selected is always the same, and if 
each of the other (i.e. undesired) choices has the same probability of selec¬ 
tion (i.e. (1 — P)/(N — 1)), then bit rate, or bits/trial ( B ), is: 
B = log 2 N + P log 2 P + (1 - P)log 2 [( 1 - P)/(N - 1)]. For each N, bit 
rate is shown only for accuracy > 100/A/ (i.e. >:chance) (from Wolpaw 
et al., 2000a). 
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literally twice as good as 80%, and only half as good as 
100%. Bit rate is an objective measure for measuring 
improvement in a BCI and for comparing different BCIs. 
It can also help select applications (Wolpaw et al., 2000a). 

4.9.4. Evaluation in relevant situations 

BCI evaluation should also include testing in circum¬ 
stances like those of real-life applications. As noted 
above, assessment of online performance is essential. In 
addition, BCIs should be tested under conditions in which 
the user chooses the message or command. This testing can 
reveal how well the BCI adapts to spontaneous variation in 
the signal features when it does not have the advantage of 
knowing what the output is supposed to be. It is also impor¬ 
tant to evaluate how well BCI operation combines with 
other brain functions. A BCI that requires total user atten¬ 
tion might not support a conversation or other interaction in 
which the user must continually choose the messages to 
send and evaluate the responses they elicit from the other 
person or from an external device. 

4.9.5. A general purpose BCI system for research and 
development 

In order to facilitate and encourage accommodation to 
these important principles across BCI research efforts, 
several groups are currently collaborating on development 
of a general purpose BCI system, called BCI2000 (Wolpaw 
et al., 2000c; Schalk et al., 2001). The rationale for this effort 
is that, while different BCI systems differ widely in their 
signal features, feature extraction methods, translation algo¬ 
rithms, output devices, and operating protocols, they all need 
these basic components (Fig. 1). BCI2000 is a documented, 
distributed, and open general purpose BCI system, with 4 
interacting processes: signal acquisition and storage; feature 
extraction and translation; device control; and operating 
protocol. Each process is independently executable in 
Windows NT/2000 on the same machine or on several 
networked machines. The processes interact through a prede¬ 
fined interface, so that different versions of each are inter¬ 
changeable, and so that different languages can be used. The 
plan is to make BCI2000, with associated data storage and 
analysis tools, available to those engaged in BCI research and 
development. The goal is to facilitate progress and promote 
use of standard methods for evaluating performance. 

4.9.6. Documentation and dissemination of results 

Finally, if the recent interest and progress in BCI research 

is to develop into a stable and successful research endeavor, 
the focus must be on production of peer-reviewed primary 
articles in high-quality scientific and engineering journals. 
Furthermore, researchers should recognize that the intense 
and often distorted media attention that the idea of direct 
brain-computer communication attracts, while an advantage 
in some respects, is also a problem, because it engenders 
unrealistic expectations in the public and skepticism in 
scientists. Thus, it is important for researchers to be conser¬ 


vative in their interactions with the media, and to adhere as 
closely as possible to ‘the Ingelfingerrule’, the principle that 
peer-reviewed publication should precede any other detailed 
dissemination of research results. While studies are often 
first reported in meeting presentations and abstracts, and 
may reach the popular media in that way, their first complete 
description and documentation should be in a peer-reviewed 
format (Reiman, 1981). 


5. Conclusions 

A BCI allows a person to communicate with or control 
the external world without using the brain’s normal output 
pathways of peripheral nerves and muscles. Messages and 
commands are expressed not by muscle contractions but 
rather by electrophysiological phenomena such as evoked 
or spontaneous EEG features (e.g. SCPs, P300, mu/beta 
rhythms) or cortical neuronal activity. BCI operation 
depends on the interaction of two adaptive controllers, the 
user, who must maintain close correlation between his or her 
intent and these phenomena, and the BCI, which must trans¬ 
late the phenomena into device commands that accomplish 
the user’s intent. 

Present-day BCIs have maximum information transfer 
rates <25 bits/min. With this capacity, they can provide 
basic communication and control functions (e.g. environ¬ 
mental controls, simple word processing) to those with the 
most severe neuromuscular disabilities, such as those locked 
in by late-stage ALS or brainstem stroke. They might also 
control a neuroprosthesis that provides hand grasp to those 
with mid-level cervical spinal cord injuries. More complex 
applications useful to a larger population of users depend on 
achievement of greater speed and accuracy, that is, higher 
information transfer rates. 

Future progress hinges on attention to a number of crucial 
factors. These include: recognition that BCI development is 
an interdisciplinary problem, involving neurobiology, 
psychology, engineering, mathematics, computer science, 
and clinical rehabilitation; identification of the signal 
features, whether evoked potentials, spontaneous rhythms, 
or neuronal firing rates, that users are best able to control; 
the extent to which this control can be independent of activ¬ 
ity in conventional motor output and sensory input channels; 
the extent to which this control depends on normal brain 
function; identification of the best feature extraction meth¬ 
ods and the best algorithms for translating these features 
into device control commands; development of methods 
for maximizing each user’s control of these signal features; 
attention to the identification and elimination of artifacts 
such as EMG and EOG activity; adoption of precise and 
objective procedures for evaluating BCI performance; 
recognition of the need for long-term as well as short-term 
assessment of performance; identification of appropriate 
applications; proper matching of BCI applications and 
users; close attention to factors that determine user accep- 
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tance of augmentative technology; and emphasis on peer- 
reviewed publications and appropriately conservative 
response to media attention. With adequate recognition 
and effective engagement of these issues, BCI systems 
could provide an important new communication and control 
option for those with disabilities that impair normal commu¬ 
nication and control channels. They might also provide to 
those without disabilities a supplementary control channel 
or a control channel useful in special circumstances. 
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